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Abstract 


Form-Focused Instruction for the English Article: 
A Meta-Analysis 


Andrew Schenck - Wonkyung Choi 


Factors, such as treatment complexity and duration, language proficiency, and 


educational context (EFL vs. ESL), can significantly influence language performance 


in experimental studies. The purpose of the present meta-analysis is to investigate 


the degree to which these causal factors influence linguistic improvement in 


studies of the English article. Nineteen experimental studies of the English article 


were chosen for analysis. After data were collected, the effect sizes of individual 


causal factors were compared using Spearman rank-order correlation. Next, the 


combined influence of the causal factors was calculated by performing a multiple 


regression. The results suggest that each hypothesized influence contributes to the 


effect size, and together, they account for nearly one fifth of the variance of 


effect sizes within individual studies. Due to the significance of the results, it 


appears essential that the determinants examined in this study be considered when 


assessing the effectiveness of grammar-focused instructional techniques. In the 
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future, more multivariate, holistic studies are needed to understand how systematic 


differentiation of these multiple causes can improve the acquisition process. 


Keywords: Meta-analysis, English Article, Treatment Complexity, Treatment 
Duration, Language Proficiency, EFL, ESL 
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I. Introduction 


Second language researchers and educators have traditionally been polarized 
into two camps, one emphasizing the utilization of purely meaning-focused 
activities, and another supporting an explicit focus on grammatical accuracy. In 
the past, proponents of an explicit focus on accuracy advocated the use of 
“synthetic” syllabi that emphasized grammatical structures over the contexts in 
which these structures were used. The syllabi were used with the Grammar- 
Translation and the Audiolingual approaches, which promoted grammar and 
vocabulary learning through rote memorization and language drills. As stated by 
Long and Robinson (2004), this type of emphasis on grammar was often 
ineffective, because it utilized an antiquated notion that language is a process of 
habit formation. It treated language as an academic exercise that could be 
learned, rather than an internal cognitive process which is molded by external 
stimuli. As a result, second language learners educated through such methods 
often gained a conscious understanding of grammatical rules and vocabulary, but 
could not effectively communicate through either oral or written discourse. 

In contrast to grammar-based methods of language pedagogy, techniques which 
focused on meaning, such as the Natural Approach or Language Immersion, 
provided authentic input and contexts that promoted communication. Advocates of 
this teaching style asserted that grammar instruction was unnecessary and 


ineffective, since learners acquired grammar through an internal cognitive process 
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(Krashen & Terrell, 1983). Theories such as the Natural Order Hypothesis and 
Processability Model, which each outlined seemingly invariant orders of 
grammatical acquisition, were used to bolster this claim (Dulay, Burt, & Krashen, 
1982; Dyson, 2009; Makino, 1979; Pienemann, 1991, 2005; Simmons, 2001). 

While acquisition orders appeared to be highly immutable and _ purely 
cognitive, more modern research suggests that the sequences are indeed variable 
and molded by external environmental factors such as input frequency and 
salience (Goldschneider & DeKeyser, 2005; Ellis, & Collins, 2009; Ellis, & 
Larsen-Freeman, 2009; Luk & Shirai, 2009). Clearly, there are some external, 
environmental factors that influence the acquisition process, suggesting that input 
and pedagogical reform should be used to enhance the grammatical accuracy. The 
need for such reform is apparent when considering the highly meaning-focused 
immersion programs implemented in Canada. Although learners within these 
programs were provided with large amounts of input and ample opportunities to 
communicate, they continued to make basic errors with morphology and syntax 
(Williams, 1995). Concerning this issue, Williams (1995) stated that, “in focusing 
exclusively on meaning and the overall success of communication, we have 
overlooked the issue of accuracy.” (p. 13). As implied by this statement, there is 
a distinct need for emphasis of grammatical accuracy within communicative 
lessons. 

While debates over the importance of meaning or accuracy continue, this 
controversy is much less contentious than it used to be (Renandya, 2013). The 
importance of integrating form-focused instruction within communicative lessons is 
now well documented (Norris & Ortega, 2000; Spada & Tomita, 2010). Although 
research has been able to establish a need for explicit grammar instruction in a 
general sense, it has still failed to clearly outline how pedagogical interventions 
and treatments should be tailored to the unique characteristics of each 
grammatical feature. This is primarily because research has sought to establish 
all-purpose pedagogical frameworks for grammatical features, rather than 
differentiated approaches based upon distinctive morphological or syntactic 


characteristics. 
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It appears that differentiation of instruction for grammatical features has been 
largely hampered by insufficient analysis of individual morphosyntactic differences 
and their impact on the instructional process. Although the meta-analysis by 
Spada and Tomita (2010) did examine grammar-focused instructional differences 
based on the complexity of morphosyntactic features, calculations were overly 
simplistic (complex vs. simple feature designations) and were based upon a 
linguistic concept of transformations which has been largely discounted 
(Goldschneider & DeKeyser, 2005). Goldschneider and DeKeyser (2005) have 
recognized that determinations of grammatical difficulty are more accurately based 
upon multiple factors that include complexity of form, complexity of meaning, 
and complexity of form-meaning mapping (DeKeyser, 2005). Because these factors 
determine how and when a grammatical feature is acquired, it is essential that 
they be collectively utilized to ascertain the efficacy of grammar-focused 
instruction. It is also important that influences such as language proficiency and 
educational context (EFL vs. ESL context) be considered, since they may impact 
a learner’s ability to understand and benefit from pedagogical emphasis of a 
target feature. Through holistically investigating multiple influences, educators may 
better understand what types of instructional techniques are most effective for 
each grammatical feature, and in what educational contexts these techniques 
should be used. 


II. Literature Review 


1. Form-Focused Instruction 


As with studies of other grammatical features, Form-Focused Instructional 
(FFI) research of the English article has expanded our understanding of how 
acquisition may be enhanced (Bitchener, Young, & Cameron, 2005; Ellis, Sheen, 
Murakami, & Takashima, 2008; Jang, 2006; Master, 1997; Sheen, 2010). 
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Bitchener (2008), for example, found that both written and oral corrective 
feedback could significantly improve use of the article. Sheen (2010), likewise, 
found that metalinguistic explanations and direct feedback could enhance written 
use of the same target feature. In regards to reading and listening input, Wong 
(2001) discovered that simultaneous emphasis of the definite article and passage 
content reduced performance on the target feature, suggesting that a more focused 
approach to FFI was needed. Subsequent research has confirmed the accuracy of 
this assertion (Ellis, Sheen, Murakami, & Takashima, 2008; Sheen, Wright, and 
Moldawa, 2009). 


2. The English Article 


In addition to studies exploring pedagogical reform of input and output, 
research has been conducted to determine how sociological aspects of explicit 
grammar instruction may be utilized to enhance learning of the English article. 
Muranoi (1996, 2000), for example, studied how interactive feedback, in the form 
of negative recasts and explicit grammar explanation, could help student leaders 
within task-based groups. The study revealed that explicit debriefing of the target 
grammar yielded the most significant effect. Nassaji and Swain (2010) also 
studied sociocultural factors of explicit grammar instruction. Using a Vygotskian 
scale ranging from explicit to implicit strategies of interaction, one learner was 
provided with assistance appropriate for their Zone of Proximal Development 
(ZPD), while another learner was provided with random interventions. Results 
revealed that the learner who received level-appropriate ZPD assistance utilized 
articles correctly 82.8% of the time, while their counterpart used them correctly 
only 40% of the time. 

Although we have gained a more holistic understanding of FFI designed for 
the English article, the knowledge gained from individual studies cannot easily be 
compared or contrasted. This is due to methodological differences of treatments 
which influence results. In past studies of English articles, researchers used 


widely different treatments to examine grammar instruction. Some of the 
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treatments, such as those by Master (n.d., 1994), for example, used complex 


interventions that emphasized 4 types of lexical variants (a, an, the, 0) and 14 to 
20 different categorical distinctions of English article (See Table 1). 


Table 1. Scope of Treatments Within Studies of the English Article 


Bitchener (2008) Muranoi (1996) 
pady Bitchener & Knoch (2008)|Muranoi (2000) DASE ete 
& * 6699 I fi it : F 
Target Fish neon de Bee Definite and Indefinite 
and subsequent article ; 
Structure . : : Articles 
mention “the” (first mention) 
A, An, The, 0 
A A > 2 : > 
Forms ate eae (Zero Article) 
-Classification (e.g., “a book 
is what you read with’) 
-Identification (e.g., “the 
milk on the table is yours”) 
-Count (e.g., “a book’) 
; -Noncount (e.g., “dinner’) 
sou Sigua Noun -First Mention (e.g., “a man’) 
(e.g. “A pencil”) -Subsequent Mention (e.g., 
Number of ; : . “A man went into the 
. -Unique by prior utterance/Count Singular Bho oat 
Semantic / . : «, |store. The man said...”) 
; (e.g, “A man went into/Noun (eg. “A Ate Lae 
Syntactic : ae -Descriptive adjectives (e.g., 
: the store. The man sai/pencil’) Wee ‘s 
Categories d...”) a first step”) 
‘+ -Ranking adjectives (e.g., 
“the first step”) 
-Defining relative clause 
(e.g., “a car that goes 50 
mph”) 
-Limiting relative clause 
(e.g., “the car that won the 
race” 


Form-Focused Instruction for the English Article: A Meta-Analysis 67 


-Partitive “of’ phrase (e.g., 
“a cup of coffee’) 

squirrel has a tail”) 
-Descriptive “of” phrase 
(e.g., “the temperature of 
the sun’) 

-Generic “the” (e.g., “the 
squirrel is becoming a pest”) 
-General characteristics 


(e.g., “a 
Number of -New knowledge (e.g., “a 
Semantic / catastrophe”) 
Syntactic -Shared knowledge (e.g., 
Categories “the moon’) 


-Proper nouns (e.g., “the 
Amazon River’) 
-Classified proper nouns 
(e.g., “a Mr. Jones is here 
to see you”) 

-Identified proper nouns 
(e.g., “the Mr. Jones you 
met last night’’)Idiomatic 
phrases (e.g.,“have a cow,” 
“rise to the occasion) 


Combined 

Complexity 
(Variants + 
Categories) 


Other studies of corrective feedback, in contrast, only chose to explicitly 
emphasize two lexical variants (a, the) with two categorical distinctions (a for 
first mention and the for subsequent mention) (Bitchener, 2008; Bitchener & 
Knoch, 2008). Because widely different forms of treatment were placed under one 
umbrella term, “form-focused instruction,” disparities among these treatments were 


largely overlooked. 
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3. Research Questions 


Questions concerning both treatments (complexity and duration) and indirect 
influences of treatments (learner proficiency and educational context) were 
constructed to assess the impact of form-focused instruction on language 
proficiency. The following questions about experimental studies of the English 
article were posed: 

1. To what extent is language performance affected by the complexity and 
duration of form-focused instruction? 

2. To what extent is language performance affected by language proficiency 
of the participants? 

3. To what extent is language performance affected by educational context 
(ESL vs. EFL)? 

4. How do factors of treatment, language proficiency, and educational context 
combine to predict language proficiency within experimental studies? 

While the first three questions were designed to analyze influences of 
individual variables, the final question was crafted to collectively examine the 


impact of all variables. 


Ill. Method 


1. Data Collection 


The purpose of this meta-analysis was to examine the influence of treatment 
complexity, the number of treatments, language proficiency, and context (EFL vs. 
ESL) on language performance within experimental studies of the English article. 
To select studies for examination, university library databases and Google scholar 
were systematically explored using search strings which included "definite article" 


or “indefinite article” and additional search strings such as "focus on form’, 
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Cn 


“control group”, “processing instruction”, or “dictogloss”. References found within 
experimental studies of the English article or other meta-analyses (e.g., Kao, 
2013) were also systematically searched and included when applicable. Following 
the investigation, a pool of over 40 studies of English article instruction was 
available for evaluation. Since not all of these studies included the necessary 
elements for study, they were assessed for inclusion using the following criteria: 

1. Includes form-focused treatment of the English article 

2. Includes information about treatment complexity and duration 

3. Has a control group for calculation of effect size 

4. Includes an assessment of language performance 

Using the above criteria, 19 studies were deemed eligible for inclusion within 
the meta-analysis. These studies are denoted with an asterisk (*) in the reference 


section of the paper. 


1. Independent Variables 


After research was chosen for the meta-analysis, data for each independent 
variable was systematically collected from the selected studies and recorded within 
a database. Information about the following independent variables was recorded: 

1. Treatment Complexity 

2. Treatment Duration (Number of Sessions) 

3. Learner Proficiency 

Treatment complexity was determined by adding the number of lexical 
variants included within the emphasis (total of four possible variants a, an, the, 0 
[the zero article]) and the number syntactic and/or semantic categories. Studies by 
Bitchener (2008) and Bitchener & Knoch (2008), for example, emphasized two 
lexical variants (a, the) with two categorical distinctions (a for first mention and 
the for subsequent mention), meaning that there was a treatment complexity of 4 
(2 lexical variants + two categorical distinctions). Additional information about the 


categorical distinctions within studies may be viewed in Table 1. In addition to 
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treatment complexity, treatment duration was recorded as the number of treatment 
sessions utilized within a study. 

Learner proficiency was recorded on a scale from 1 to 9. Number 1, for 
example, represented Novice Beginner, 2 represented Novice Mid, and 3 
represented Novice Upper. Research designations as intermediate low, for example, 
were recorded as 4 (Intermediate Mid). If only beginner, intermediate, or 
advanced levels of proficiency were designated, the Mid value was used. 
Designations of language proficiency within these studies were not based on one 
reliable instrument. However, it was thought that the rudimentary classifications 
may reveal patterns that could be further investigated within future research. 
Although insightful, the proficiency levels within these studies should be 
interpreted with caution. 

Educational context was represented as a binary value. The value 1 


represented an EFL context, while the value of 0 represented an ESL context. 


2. Dependent Variable and Effect Size 


Scores on language performance tests within the selected studies served as the 
dependent variable for the meta-analysis. These tests were a mixture of 
instruments that assessed both explicit (conscious) and implicit (unconscious) 
knowledge. While separation of these two test types for evaluation was desirable, 
the ability to do this was precluded by many experimental studies, which used a 
combination of both types of knowledge to assess learner performance. It was 
thought that comprehensive evaluation of all instruments would provide insight 
that could sponsor further inquiry at a later time. 

Language performance scores were operationally defined by calculating effect 
size. Before performing this calculation, test score information for each 
experimental group and control group was entered into SPSS Version 20. All 
groups included within an experimental design were included within the database. 
Information for calculating effect size, such as posttest averages and their standard 


deviations, were all recorded for the experimental and control groups. Following 
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data entry, effect size for posttest mean scores was calculated using the method 
employed by Spada and Tomita (2010, p. 307): 

d = [M-M))/[SQRT[(SD)SD\|+SD2SD>]/2] 

If information about posttest scores was unavailable, and information about the 
percentage of improvement was available, effect size was determined using an 
arcsine transformation chart (Lipsey & Wilson, 2001, p. 204). No other effect 
size calculations were required for this study. After effect size was computed, a 
weighted effect size was then calculated. The following formula was used for this 
calculation (Spada & Tomita, 2010, p. 308): 

d@’=[1-3/(4N- 9) d 

The utilization of weighted effect size helped to ensure that studies with a 
small number of participants did not unnecessarily bias results within the 


meta-analysis. 


3. Comparing Independent Variables with Effect Size 


Statistical comparison of independent variables to the dependent variable 
(effect size) was performed in two steps. First, both unweighted and weighted 
effect sizes were compared to treatment complexity, treatment duration, leamer 
proficiency and educational context using the Spearman’s rank correlation. This 
calculation was performed to assess individual influences of each independent 
variable in accordance with research questions one through three. Second, 
treatment complexity, treatment duration, learner proficiency, and educational 
context were compared to the dependent variable, effect size, using the multiple 
regression formula. This computation was an attempt to measure the collective 


influence of all 4 independent variables in accordance with research question four. 
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IV. Results and Discussion 


1. Treatment Complexity, Treatment Duration, and Effect Size 


Correlations between treatments of experimental studies and effect sizes 
yielded interesting results. As can be seen in Table 2, both the number of 
concepts explicitly covered and duration of treatments were significantly related to 
effect size (See Table 2). 


Table 2. Spearman’s Rank Correlations to Effect Size and Weighted Effect Size 


Effect Size Weighted Effect Size 
Correlation 4 ae 
Number Of Coefficient “916 “lt 
Explicit 
Concepts Sig. (2-tailed) .000 .000 
Covered 
N 73 73 
Correlation oo we 
oe -.345 -.343 
Number Of Coefficient 
Treatment : : 
Seasionis Sig. (2-tailed) .003 .003 
N 73 73 


The number of concepts covered within treatments yielded the highest 
correlation. The relationship was highly significant for both unweighted (7, = 
-.516; p = .00) and weighted effect sizes (7; = -.514; p = .00). The high, 
negative correlation suggests that as the number of lexical variants and categories 
within a treatment increases, student performance on language tasks tends to 
decrease. Such an assertion is supported by more detailed investigation of studies 
included within this meta-analysis. Figure 1, for example, which graphically charts 
the effect sizes for each level of treatment, reveals a clear decline as complexity 


increases. 
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Average Weighted Effect Size 


3 4 5 ? g "1 12 17 Ll] 19 ™M 3% 


Number Of Explicit Concepts Covered in Treatment 


Figure I. Comparison of average effect size to scope of treatment. 


Like Figure 1, examination of individual studies revealed a_ relationship 
between higher treatment complexity and lower effect sizes. Studies by Master 
(n.d., 1994), for example, which had treatment complexities ranging from 11 to 
35, had negative weighted effect sizes which averaged -0.246. Similarly, a study 
by Wong (2001), which integrated aspects of passage content into explicit 
emphasis of the definite article (treatment complexity of 12), had a negative 
weighted effect size of —0.6625. 

Clearly, the number of concepts covered within a treatment appears to have 
some impact on overall language performance. Treatment complexity may also 
explain lower effect sizes found within studies of unfocused corrective feedback. 
A study by Sheen, Wright, and Moldawa (2009), for example, which used highly 
complex unfocused corrective feedback for multiple grammatical features (total 


treatment complexity of 9; 4 for the English article plus 5 for the other 
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grammatical features), had a negative effect size of -.01. In contrast, more 
simplistic corrective feedback focused on the English article (treatment complexity 
of 4), had a positive effect size of .98, nearly 1 standard deviation above the 
mean. 

In contrast to studies with a high treatment complexity, those with a low 
treatment complexity revealed higher effect sizes. A study of computer-mediated 
corrective feedback by Sauro (2009) had the largest effect sizes, yielding 6.21 for 
the recast group and 16.30 for the metalinguistic feedback group. Unlike more 
complex treatments, the focus of this treatment was very narrow in scope, 
covering only one lexical variant of the article (the zero article) with abstract and 
noncount nouns (total complexity of 3). This study had the lowest instructional 
complexity of all treatments within the meta-analysis and the highest effect size. 
Other simplistic treatments, such as that by Bitchener and Knoch (2008), which 
covered only a for first mention and the for the second mention, tended to yield 
effect sizes of 1 or higher. 

Like treatment complexity, a significant relationship between effect size and 
the number of treatment sessions was revealed through statistical analysis. 
Correlations to both the unweighted (7, = -.345) and weighted effect sizes (r,; = 
-.343) were significant at the .05 level ( = .003). Superficially, the highly 
significant negative relationship between effect size and the number of treatments 
appears to suggest that a larger number of treatments will adversely affect learner 
performance. More detailed analysis of treatment sessions, however, suggests that 
this conclusion is inaccurate. Treatment complexity was not uniform throughout 
the number of treatment sessions. While effect size was very large for studies 
that utilized just one treatment, these studies tended to be more simplistic, 


covering 5 concepts or less (See Figure 2). 
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Average Weighted Effect Size 


2 3 4 
Number Of Treatment Sessions 
Figure 2. Comparison of Average Effect Size to 

Number of Treatment Sessions 


Contrary to studies having only one treatment session, those having multiple 
sessions (from 2 to 5) included more complex instruction with a large number of 
concepts, ranging from 4 to 35. The only study which utilized 5 treatment 
sessions had a highly complicated explicit grammatical focus covering 18 concepts 
(Master, 1994). Taking into account the rise in complexity as the number of 
sessions increased, the decline in effect size is hardly surprising. Treatments with 
only a few concepts appear to be effective after one session, while those with 
more concepts require additional sessions to increase effect size. Further inquiry is 
needed to concretely determine how many treatments are required for each level 
of instructional difficulty. This inquiry will require the utilization of holistic 
designs that include treatments of varying complexity and duration. Such research 
will firmly establish how the number of concepts explicitly covered interacts with 


the number of treatment sessions to influence overall effectiveness of instruction. 
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2. Learmer Proficiency Level and Effect Size 


In regards to learner proficiency level, correlations to both the unweighted (7; 
= -.274; p = .030) and weighted effect sizes (r, = -.269; p = .033) were 
highly significant at the .05 probability level. Like the association between 
treatment sessions and effect size, the negative relationship with language ability 
may be an indirect reflection of treatment complexity. Among the studies selected 
for meta-analysis, more complicated instruction focusing on form tended to be 
given to higher proficiency learners. With the exception of only one experimental 
study, for example, groups with the largest treatment complexities, ranging from 
11 to 35 concepts, had the highest proficiency levels (from Intermediate Mid to 
Advanced). The close connection between language proficiency and treatment 
difficulty serves to explain both the significance and negativity of the Spearman’s 
rank correlation. The relationship between proficiency and effect size may thus 
reflect, albeit indirectly, a need for additional treatment sessions commensurate 
with the instructional complexity of a grammatical focus. 

It should be mentioned that while insightful, any correlation of proficiency 
level to effect size must be interpreted with caution. First of all, there is not a 
normal distribution of proficiency levels. With the exception of only three studies 
(Bitchener & Knoch, 2010a; Master, 1994; Shintani & Ellis, 2013), all other 
research projects included learners at the intermediate level (intermediate high, 
mid, or low). Such a narrow scope severely limits understanding of the 
relationship between language proficiency and effect size of FFI treatments. A 
further complication is the lack of standardization of proficiency designations. 
They are based upon opinions of the researcher, rather than on assessments 
obtained from a standardized instrument. Essentially, the lack of standardized and 
reliable information about proficiency, along with the utilization of participants 
with highly similar ability levels, makes further investigation of instruction of the 
English article necessary. If future research utilizes participants of highly variable 
proficiency levels and consistently evaluates their progress through standardized 


instruments, educators may be able to understand when a particular grammar 
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treatment should be introduced. 
3. EFL / ESL Context and Effect Size 


In contrast to proficiency level, comparison of effect size to ESL / EFL 
contexts did not reveal a significant difference. The Spearman’s rank correlations 
for both unweighted (7; = .044; p = .713) and weighted (7; = .045; p = .704) 
effect sizes were very low, suggesting that context may not have a significant 
impact on English article instruction. 

Although insignificant, mean effect sizes for EFL and ESL contexts seemed 
considerably different. Posttest scores within EFL contexts were higher, yielding 
an average weighted effect size of 1.678, while the average ESL context effect 
size was .659. The difference may indicate that there is some influence within 
EFL contexts, albeit insignificant, which makes either explicit or implicit grammar 
treatments more effective. If an influence does indeed exist, it may signify that 
learners in EFL contexts, lacking sufficient input to acquire a foreign language 
naturally, rely more heavily on explicit grammar reflection and metalinguistic 
strategies to learn a language. More research is needed to better understand if, 


and in what way, educational context influences grammar instruction. 
4. Integrating Variables and Predicting Effect Size 


Results of the multiple regression formula suggest that independent variables 
(treatment complexity, treatment duration, proficiency level, and ESL/EFL context) 
collectively serve as one significant predictor of effect size. While the regression 
correlation was R = .436, the R Square value was R’ = .19, indicating that 
nearly 20% of variance in effect size can be explained through concomitant 
consideration of the hypothesized causal factors. 

While the multiple regression correlation is not high, the corresponding 
ANOVA value (F = 3.411; p = .014) does suggest that this value is significant, 


and could not occur by chance. In addition to this significance, low variance 
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inflation factors (VIF), all below 5, and high tolerance levels, all above .2, 
further suggest that there is no multicollinearity between the independent variables 
(Appendix A). Taken as a whole, statistical analysis appears to indicate that each 
of the independent variables has a distinct role in the prediction of overall 


language performance. 


Table 3. Multiple Regression 


Adjusted R Std. Error of the 
Model R R Square ; 
Square Estimate 


While the value generated by the regression formula cannot predict effect 
sizes in their entirety, it is large enough to explain a fifth of the variability 
between studies. Clearly, there is a substantial influence exerted on language 
performance by treatment complexity, the number of treatments, language 
proficiency, and educational context. Although plainly evident within the current 
meta-analysis, past research has not clearly identified this relationship. The failure 
appears to be due to overemphasis of a scientific method that regulates 
“extraneous” experimental conditions through the maintenance of uniformity (e.g., 
same treatment complexity, same number of treatment sessions, participants at 
similar levels of proficiency, etc.). Despite being insightful, research conducted in 
this way can only answer if a particular pedagogical technique is effective under 
the unique circumstances of an individual study. Such a limited scope does not 
provide the information necessary for real-world contexts, which include a number 
of consistently changing variables; instead, it merely provides vague answers to 
simplistic questions such as: 

1. Is form-focused instruction effective? 

2. Are recasts effective? 


3. Is implicit (or explicit) instruction more effective? 
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Rather than simply controlling for a few independent variables through 
making other experimental conditions the same, new research must focus on the 
methodic manipulation of both independent and environmental variables. Aspects 
of treatment complexity and duration, for example, should be systematically 
changed to identify just how useful a particular treatment is. Likewise, proficiency 
level should be systematically modified to determine when a particular type of 
grammar instruction has the optimal effect. Such systematic variation in 
experimental studies will help provide a more holistic perspective of form-focused 
instruction and its impact on the acquisition process. Moreover, it may serve to 
answer more in-depth questions such as: 

1. To what degree is a particular pedagogical technique useful? 

2. When should the pedagogical technique be introduced (e.g., at what 
proficiency level)? 

3. Which grammatical feature benefits most from this pedagogical technique? 

New research resolving questions such as these will provide more detailed 
guidance for curriculum design and instruction. Unlike past experimental studies 
examining only one independent variable in isolation, multivariate approaches may 
yield pragmatic guidelines for the highly variable real-world contexts in which 
they must be applied. 

One major issue concerning systematic differentiation of multiple variables is 
that the more holistic research objectives required for such investigation cannot 
easily be fulfilled within just one study. Often, several separate studies must be 
conducted and subsequently compared to achieve research goals. In order for this 
to be plausible, larger cooperative efforts using standardized measures for 
variables such as treatment complexity, treatment duration, proficiency level, and 
educational context will be needed. Standardization in this way can allow for the 
controlled collection of data, methodical comparison of results, and development 
of new, more holistic perspectives. As results of experimental studies are 
compared and compiled into a more comprehensive framework, an empirically 
testable curriculum may be developed which allows educators to systematically 


“engineer” grammatical accuracy. In the future, more holistic study may also 
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yield algorithms for automated technologies that greatly enhance the acquisition 


process. 


V. Summary and Conclusions 


The present study sought to investigate whether multivariate analysis of 
treatment complexity, treatment duration, proficiency level, and _ instructional 
context (EFL vs. ESL) could explain language performance within experimental 
studies of the English article. Spearman’s rank correlations of treatment 
complexity revealed significant negative relationships to both unweighted and 
weighted effect sizes, suggesting that language performance was closely linked to 
the difficulty of grammar-focused instruction. As the lexical variation and 
semantic concepts of grammatical instruction increased, performance of the 
English article tended to decrease. While the results appear to suggest that very 
complex treatment is ineffective, it may actually indicate that more treatment 
sessions are needed. Additional research is required to identify the relationship 
between treatment complexity and treatment duration, so that the efficacy of 
form-focused instruction for the English article may be enhanced. 

Although treatment sessions and language proficiency were both significantly 
correlated to effect size, these relationships were weaker than the link reported 
between treatment complexity and effect size. In actuality, the negative 
correlations for treatment sessions and language proficiency appeared to be 
indirect reflections of treatment complexity. More difficult treatments were 
generally given to higher proficiency learners through more _ extensive, 
time-consuming interventions. While ESL/EFL context was not significantly 
correlated to effect size, educational environment may still be a_ partial 
determinant of language performance. This view is supported by multiple 
regression analysis, which revealed that context was the most significant predictor 
of effect size (Appendix A). 


Pro- 
ficiency 
Level 


Beginner 


Inter- 
mediate 


Ad- 
vanced 


On the 
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Table 4. Sample Emphasis for the English Article Based 
on Treatment Complexity and Proficiency 


Semantic / Syntactic 
Concepts for Explicit 
Presentation(Top-down / 
Focus on Meaning) 


“uniqueness” 
first mention / 
subsequent mention 
abstract nouns / unique 
things in our situation / 
unique things in our 
neighborhood / 
unique things in our city / 
parts of a list 


“uniqueness”, first mention, 
subsequent mention, 
noncount nouns 

unique things in our 
world / abstract nouns / 
noncount nouns / 

unique things in our 

society / unique situation / 
unique things in our 
neighborhood / 

unique things in our city / 
elements of a list 

generic things / 

unique things in our world / 
unique things in our society 
/ unique situation / 
unique in our neighborhood 
/ unique things in our city / 
abstract nouns / 

noncount nouns 


whole, multiple regression analysis indicated 


Forms for 
Explicit 
Presentation 
(Bottom-up/ 
Focus on form) 
the 
a, the 


the 


a, an, the, 0 


the, 0 


a, an, the, 0 


Difficulty 
Level of 
Explicit 
Instruction 


I+ 1 =2 
2+3= 


6+3= 
12 


that 


Duration 
in Hours 


combined 


consideration of the independent variables could explain nearly 20% of the 
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variation in effect sizes of experimental studies. The significance of these results 
suggests that the aforementioned independent variables must be considered when 
ascertaining overall efficacy of experimental treatments. Furthermore, they should 
be systematically utilized within grammar curricula. The curriculum in Table 4 is 
just one example of how these multiple factors may be exploited to improve 
form-focused instruction of the English article (See Table 4). 

Unlike most designs in modern ESL/EFL textbooks, the explicit concepts in 
Table 4 simultaneously consider aspects of treatment complexity, treatment 
duration, and language proficiency. As grammar-focused concepts grow in 
complexity, they are utilized with higher proficiency learners in a duration which 
is commensurate with difficulty level. Through using curricular frameworks like 
that in Table 4, researchers can both consider and analyze multivariate influences 
within their experimental designs, and educators can utilize the multiple variables 
to systematically and incrementally improve instruction. 

While the conceptual framework in Table 4 is a step forward, more research 
is needed to confirm that the multiple variables are correctly utilized. In order to 
study multivariate relationships in a holistic way, several studies will need to be 
conducted and compared using standardized measures for treatments, proficiency 
levels, and educational contexts. Future studies must look beyond simplistic 
designs that analyze just one independent variable in isolation. Such research is 
far too limited to provide pragmatic solutions for highly complex real-world 
issues. Research projects must now begin to develop holistic, multi-stage designs 
which investigate not only numerous variables, but the systematic modification of 
these variables and their associated relationships. Such a methodological shift will 
help researchers and educators develop holistic curricular frameworks that can 


engineer desired results and significantly enhance the acquisition process. 
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Appendix A 


Table 1. ANOVA Results 


Mean Square 


Squares 
Regression 57.645 
1 |Residual 245.057 58 4.225 
Total 302.703 
a. Dependent Variable: Weighted Effect Size 


b. Predictors: (Constant), EFL/ESL Context, Number Of Explicit Concepts Covered, 
Proficiency Level, Number Of Treatment Sessions 


Table 2. Collinearity Statistics 


Model Unstandardized | Standardized t Sig. Collinearity 
Coefficients Coefficients Statistics 
B Std. Beta Tolerance VIF 
Error 
Constant 1.409 999 1.411 164 
Number 
Of 
Explicit -.060 .047 -.180 -1.279 | .206 .706 1.415 
Concepts 
Covered 
Number 
Of 
T -.608 353 -.256 -1.723 | .090 .632 1.581 
reatment 
Sessions 
Proficiency: ||. 470% || 2297 101 749 | 457 | 772 ‘| 1.295 
Level 
EFL or 
ESL 1.908 742 318 2.569 | .013 O11 1.098 


a. Dependent Variable: Weighted Effect Size 
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